{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Query cohere-gpt-xlarge Model from AWS Bedrock \n",
    "\n",
    "\n",
    "Cohere builds a collection of Large Language Models (LLMs) trained on a massive corpus of curated web data. Powering these models, our infrastructure enables our product to be deployed for a wide range of use cases. The use cases we power include generation (copy writing, etc), summarization, classification, content moderation, information extraction, semantic search, and contextual entity extraction\n",
    "\n",
    "This sample notebook shows you how to query [cohere-gpt-xlarge](https://aws.amazon.com/marketplace/pp/prodview-r6zvppobprqmy) using Amazon Bedrock.\n",
    "\n",
    "> **Note**: This is a reference notebook and it cannot run unless you make changes suggested in the notebook.\n",
    "\n",
    "## Contents:\n",
    "1. [Subscribe to Amazon Bedrock](#1.-Subscribe-to-Amazon-Bedrock)\n",
    "2. [Perform real-time inference](#2.-Perform-real-time-inference)\n",
    "   1. [Create a Cohere AWS Client](#A.-Create-a-Cohere-AWS-Client)\n",
    "   2. [Create input payload](#B.-Create-input-payload)\n",
    "   3. [Perform real-time inference](#C.-Perform-real-time-inference)\n",
    "   4. [Visualize output](#D.-Visualize-output)\n",
    "   5. [Stream output](#E.-Stream-output)\n",
    "3. [Sample use cases](#3.-Sample-use-cases) \n",
    "   1. [Writing a product description with co.generate](#A.-writing-a-product-description-with-cogenerate)\n",
    "   2. [Writing a blog post with co.generate](#B.-writing-a-blog-post-with-cogenerate)\n",
    "   3. [Writing a cold outreach email with co.generate](#C.-writing-a-cold-outreach-email-with-cogenerate)\n",
    "4. [Clean up](#4.-Clean-up)\n",
    "   1. [Unsubscribe from the listing (optional)](#A.-Unsubscribe-from-the-listing-(optional))\n",
    "\n",
    "## Usage instructions\n",
    "You can run this notebook one cell at a time (By using Shift+Enter for running a cell)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Subscribe to Amazon Bedrock"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Follow the instructions in the [Amazon Bedrock](https://console.aws.amazon.com/bedrock) console.\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Perform real-time inference"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you want to understand how real-time inference with Amazon Bedrock works, see the [Amazon Bedrock Documentation](https://docs.aws.amazon.com/bedrock)."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Create a Cohere AWS Client"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install --upgrade cohere-aws\n",
    "# if you upgrade the package, you need to restart the kernel\n",
    "\n",
    "import cohere_aws\n",
    "import boto3\n",
    "\n",
    "co = cohere_aws.Client(mode=cohere_aws.Mode.BEDROCK)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B. Create input payload"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt = \"Write a LinkedIn post about starting a career in tech:\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C. Perform real-time inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "response = co.generate(prompt=prompt, max_tokens=50, temperature=0.9, return_likelihoods='GENERATION', model_id='cohere.command-text-v14', stream=False)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### D. Visualize output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(response.generations[0]['text'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(response.generations[0]['token_likelihoods'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### E. Stream output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "streamed_response = co.generate(prompt=prompt, max_tokens=500, temperature=0.9, return_likelihoods='GENERATION', model_id='cohere.command-text-v14', stream=True)\n",
    "for token in streamed_response:\n",
    "    print(token)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Sample use cases"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Writing a product description with co.generate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt=\"\"\"Write a creative product description for a wireless headphone product named the CO-1T, with the keywords \"bluetooth\", \"wireless\", \"fast charging\" for a software developer who works in noisy offices, and describe benefits of this product.\"\"\"\n",
    "\n",
    "response = co.generate(prompt=prompt, max_tokens=100, temperature=0.9, model_id='cohere.command-text-v14')\n",
    "\n",
    "print(response.generations[0].text)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### B. Writing a blog post with co.generate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt=\"\"\"Write a body paragraph about \"Shopify is a great case study\" in a blog post titled \"Tips from the most successful companies\"\"\"\n",
    "\n",
    "response = co.generate(prompt=prompt, max_tokens=100, temperature=0.9, model_id='cohere.command-text-v14')\n",
    "\n",
    "print(response.generations[0].text)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### C. Writing a cold outreach email with co.generate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "prompt=\"\"\"Write a cold outreach email introducing myself as Susan, a business development manager at CoolCompany, to Amy who is a product manager at Microsoft asking if they'd be interested in speaking about an integration to add autocomplete to Microsoft Office.\"\"\"\n",
    "\n",
    "response = co.generate(prompt=prompt, max_tokens=100, temperature=0.9, model_id='cohere.command-text-v14')\n",
    "\n",
    "print(response.generations[0].text)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Clean up"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### A. Unsubscribe from the listing (optional)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Steps to unsubscribe to product from AWS Marketplace**:\n",
    "1. Navigate to __Machine Learning__ tab on [__Your Software subscriptions page__](https://aws.amazon.com/marketplace/ai/library?productType=ml&ref_=mlmp_gitdemo_indust)\n",
    "2. Locate the listing that you want to cancel the subscription for, and then choose __Cancel Subscription__  to cancel the subscription.\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  },
  "vscode": {
   "interpreter": {
    "hash": "b0fa6594d8f4cbf19f97940f81e996739fb7646882a419484c72d19e05852a7e"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
